Evolution of cooperation driven by zealots 
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Abstract 



Recent experimental results with humans involved in social dilemma games suggest that coop- 
eration may be a contagious phenomenon and that the selection pressure operating on evolu- 
tionary dynamics (i.e., mimicry) is relatively weak. I propose an evolutionary dynamics model 
that links these experimental findings and evolution of cooperation. By assuming a small frac- 
tion of (imperfect) zealous cooperators, I show that a large fraction of cooperation emerges 
in evolutionary dynamics of social dilemma games. Even if defection is more lucrative than 
cooperation for most individuals, they often mimic cooperation of fellows unless the selection 
pressure is very strong. Then, zealous cooperators can transform the population to be even 
fully cooperative under standard evolutionary dynamics. 
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Humans often behave cooperatively in social dilemma situations in which withholding co- 
operative behavior is logically better. In fact, cooperation in social dilemma games has been 
explained by various mechanisms including kin selection, assortative interactions, group com- 
petition, direct reciprocity (i.e., repeated interactions), reputation-based indirect reciprocity, 
and spatial or network structure of populations [HE]- 

In contrast, recent evidence suggests that cooperation at a population level may occur as 
cascading in a social network [5J. Such a contagious view of cooperation is distinct from the 
mechanisms governing cooperation explained above, which assume that individuals at least 
try to maximize material payoffs, a central assumption in game theory including evolutionary 
game theory. In contagion, individuals may imitate others' behavior without much caring the 
material payoff to the self and others. 

The comtemporary results of the so-called upstream reciprocity (also called generalized ex- 
change and pay-it-forward reciprocity; it is a type of indirect reciprocity) are also consistent 
with contagious scenarios of cooperation. Upstream reciprocity, in which an individual helped 
by somebody helps somebody else, is widely observed in humans [1HH] . However, theory assum- 
ing payoff maximization does not support upstream reciprocity on its own [9j[Tn] . Cooperation 
on the basis of upstream reciprocity is stable only in combination with a different mechanism 
such as direct reciprocity [TT], mobility of players across groups [12], assortative interaction [15] . 
sufficiently frequent ingroup interaction [T3] . and network reciprocity [TT | [To" | [T(3] . 

Contagion of suboptimal behavior or attitudes seems to be even more common outside 
social dilemma games. Even if individuals do not like the behavior or social norm (e.g., binge 
drinking in colleges) and are not forced to adopt it, they often obey others. In sociology, such 
a phenomenon is interpreted under the framework of herd behavior, pluralistic ignorance, and 
false enforcement [T7tfT8]. 

Contagion implies that individuals change behavior under null or weak selection pressure. 
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Consistent with this, the selection pressure for humans playing the prisoner's dilemma game 
was recently shown to be relatively weak; subjects did not take the imitate-the-best behavior 
with a probability of « 30% PU (also see [H]) . 

Nevertheless, even under weak selection pressure, unconditional defection remains the unique 
Nash equilibrium of the social dilemma game. Then, how can we explain cascades of coopera- 
tion found in experiments? 

In mimicry-guided opinion formation models in which the two competing opinions are 
equally strong, a small number of zealot voters can attract nonzealous players to the pre- 
ferred opinion of the zealot Motivated by these studies, I show that a small fraction of 
zealous cooperators can reliably induce cooperation at a population level. It should be noted 
that opinion formation models and (social dilemma) games are fundamentally different in that 
only the latter involves strategic interactions and natural selection. Examples of situations in 
which zealous cooperators in social dilemmas are witnessed include military services and team 
sports and perhaps charity campaigns. In the proposed mechanism, weak selection pro- 
motes cooperation. However, the selection pressure does not have to be as weak as assumed in 
other theories of cooperation (e.g., [27J|28]). The proposed mechanism does not require addi- 
tional model components such as the conformity bias [2§]I3"0"] or so-called cooperation facilitators 
that increment payoffs to cooperators, but not to defectors [3 1 J . 

Results 

I analyze the evolutionary dynamics given by Eqs. flS} and I mainly examine the prisoner's 
dilemma game described by a standard payoff matrix given by R = 1, T > 1, and S = P = 0. 
When zealous players are absent (i.e., y — 0), D is the only Nash equilibrium (i.e., x c — 0). I 
explore the possibility that cooperation is stabilized among ordinary players in the presence of 
zealous players. 
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Prisoners' dilemma with perfect zealous cooperators 

In this section, I consider the case in which the zealous players always cooperate (i.e., p — 1, 
y c = y). In this case, Eq. (jSj) is reduced to 

dx G l-x c ( (x c + y)(x c + y -Tx c )\ 

-ar = — { {1 - w)y + w TTy J ' (1) 

Because the coefficient of x 2 c in f(x c ) = (1 — w)y + w(x c + y)(x c + y — Tx c )/(1 + y) is negative 
and /(0) = (1 — w)y + wy 2 /(l + y) > 0, the dynamics has at most one internal equilibrium, 
which is stable if it exists. When f(l) > 0, i.e., 

T<1 + -, (2) 

w 

dx c /dt > (0 < x c < 1) holds true such that the only equilibrium is located at x* = 1. 
Then, all the players eventually cooperate. Equation ()2]) indicates that weak selection and the 
presence of many zealous players facilitate such full cooperation. If T > 1 + y/w, the stable 
equilibrium x* c is given by 



-wy(T - 2) + ^w 2 y 2 {T - 2) 2 + Aw{T - l)y(y + 1 - w) 

2w(T- 1) 



It should be noted that < x* c < 1. 

The equilibrium fraction of cooperators among the ordinary players is shown as a function 
of y and T in the case of relatively weak (w = 0.1) and strong (w = 1) selection in Figs. QJa) 
and[T](b), respectively. The lines represent T = 1 + y/w and separate the full cooperation phase 
and the partial cooperation phase. Figure Q] indicates that the fraction of cooperators is mainly 
determined by y/w and is larger with the weak selection than the strong selection. 

Prisoner's dilemma with imperfect zealous cooperators 

In fact, zealous players may not perfectly cooperate. Therefore, I investigate evolutionary 
dynamics given by Eqs. (jHJ) and ([9]) with p < 1 by numerically integrating them. I use the 



Euler-Maruyama integration scheme with dt = 0.01. For a fixed set of parameter values, I 
started the evolutionary dynamics from various initial conditions, i.e., (x c ,y c ) = (0.05i, 0.05j), 
where i and j are integers and 1 < i, j < 19, and confirmed that the equilibrium is independent 
of the initial condition. 

The equilibrium fraction of cooperators among the ordinary players is shown for {w,T) = 
(0.1,1.5), (0.1, 2.5), (1, 1.5), and (1, 2.5) in Figs. [2(a) [2(b), He), 12(d), respectively. When 
w = 0.1, nearly perfect cooperation is obtained unless the fraction of zealous players (i.e., y) 
and the probability that zealous players unconditionally cooperate (i.e., p) are both small. This 
is also the case when the temptation payoff T is rather large (Fig. 12(b); T = 2.5). Even when 
w = 1, a. considerable amount of cooperation (e.g., 0.4) is observed for a wide parameter range 
when T = 1.5 (Fig. 0(c)). 

To confirm that the results are not specific to the Moran type of the reproduction process, 
I also implemented the so-called pairwise comparison rule (Methods) [27 11321434] . In short, in 
the pairwise comparison rule, the probability of the strategy replacement is a sigmoid function 
of the difference between the fitness of two randomly selected players. This update rule is 
implicated in recent laboratory experiments [19]. Numerical results for the prisoner's dilemma 
game with T = 1.5 and T = 2.5 under the pairwise comparison rule with /3 = 0.5 are shown 
in Figs. [3(a) and [3(b), respectively. The results are qualitatively the same as those under the 
Moran process (Fig. [2]). In the rest of the present paper, I use the Moran process. 

The fraction of cooperators among the ordinary players for various values of T is shown in 
Fig. HI I used two large values of p (p = 0.9 and p = 1) and two small values of y (y = 0.05 
and y = 0.1). Under both weak selection (Fig. E(a); w = 0.1) and strong selection (Fig. H(b); 
w — 1), the results do not depend much on the value of p for large p. This behavior is also 
evident in Fig. [2j Therefore, the theoretical results obtained in the previous section for the 
case of perfectly cooperating zealous players (i.e., p = 1) are translated to the case of imperfect 
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zealots (i.e., p < 1) without much change. In contrast, the fraction of cooperation is sensitive 
to the density of zealots (i.e., y). 

Snowdrift game 

The emergence of cooperation owing to the combination of ordinary players and zealots is not 
restricted to the case of the prisoner's dilemma. In this section, I briefly examine the snowdrift 
game, also known as the chicken game and the hawk-dove game [3"oTf3"T| . The payoff matrix 
of a standard snowdrift game is given by i? = (3 — 0.5, T — (3, S = (3 — 1, and P = 0, where 
(3 > 1 [37]. I set (3 = 1.5 such that the stable fraction of C in the absence of zealots is given by 
(2/3 - 2)/(2/3 - 1) =0.5. 

The fraction of cooperation among the ordinary players is shown for w = 0.1 and w — 1 in 
Figs. [5](a) and[5](b), respectively. Figure [5] indicates that the fraction of cooperators is much 
larger than 0.5 in a wide parameter range, particularly when w = 0.1. 

Discussion 

I showed that a small fraction of zealous cooperators can guide cooperation of players that obey 
a standard evolutionary dynamics. The numerical results indicate that the zealous players do 
not have to be perfectly zealous cooperators. The proposed mechanism operates better when 
the selection pressure is weak and the density of zealous players is large. Although I used the 
Moran process and the replicator dynamics, the results do not qualitatively change if a different 
strategy update rule called the pairwise comparison rule is used. 

All the present results are independent of the initial condition. Therefore, if the dynamics 
is initiated from a small density of cooperation, perhaps only among zealots, cooperation can 
cascade to prevail in the population. I emphasize that the cascade can occur even if most 
players prefer defection to cooperation to some extent. My results may provide theoretical 
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underpinning of cascades of cooperation [3] and upstream reciprocity [1HS] observed in human 
subjects. In contrast, the emergence and maintenance of cooperation based on conformity 
bias [22] requires that a majority of players initially cooperates. 

Crucial assumptions underlying the proposed explanation of cooperation are the stochas- 
ticity of the dynamics and weak selection (i.e., small w). Weak selection is often employed in 
theoretical studies because, among other things, Taylor expansion on w often leads to analytical 
conditions for cooperation (e.g., [2ZII2H])- However, experiments with human subjects present 
evidence against excessively weak selection [19]. I referred to the intensity of selection equal 
to w = 0.1 as weak selection. This value of w may not be too small to violate the reality. 
In general, the effective intensity of selection depends on the payoff matrix and the strategy 
update rule as well on the w value. Nevertheless, the following simple calculus may help: the 
largest and smallest possible fitness values in the prisoner's dilemma used in this study are 
equal to 1 — w + wT and 1 — w, respectively. Therefore, w — 0.1 indicates that the ratio of the 
two fitness values for T = 2, for example, is equal to (1 — w + wT) /(1 — w) = 11/9. If this ratio 
should exceed 2, w > 1/3 is required under T = 2. Although I only examined the extreme two 
cases, i.e., w = 0.1 and w = 1, the results shown in the figures imply that much cooperation 
will be observed with w = 1/3. In fact, some cooperation is observed even with w — 1 if proper 
conditions are met (Figs. []Jb) and Etc)). 

It should be noted that I assumed that players, either ordinary or zealous, have the same 
strength of influence on others. Although the heterogeneity in the influence of individuals, i.e., 
power, would shape collective behavior of humans, the present contribution is not about the 
power but about the relationship between zealots, weak selection, contagion, and cooperation. 

I did not ask the origin of zealous cooperators. Trivially, they will not emerge as a result 
of evolution unless other games or dynamics are simultaneously considered. One interpretation 
of this assumption is that zealous players are not interested in maximizing the material payoff. 
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Zealous cooperators are found in some real situations [26J. Another interpretation is that 
zealots are payoff maximizers but have different payoff functions. In theory of collective action, 
heterogeneity in interests and resources of individuals are suggested to elicit collective action 
to solve the free rider problem (see |38j for a review). Although the present mechanism is 
independent of that of collective action, zealots may perceive payoffs differently from ordinary 
players such that cooperation may not incur social dilemma for zealots. 

Methods 
Model 

I consider evolutionary dynamics of an infinite well-mixed population in which each pair of 
players is involved in the symmetric two-player two-strategy game once per generation. The 
payoff matrix is defined by 



C 
D 



C D 
/ \ 



(4) 



R S 

where the entries of Eq. (j3J) represent the payoffs that the row player gains. Each row (column) 
corresponds to the action of the row (column) player, i.e., cooperation (C) or defection (D). 

I assume two types of players. A player of the first type, called ordinary player, obeys a 
standard evolutionary dynamics described below. A player of the second type, called zealous 
player, may obey the evolutionary dynamics or unconditionally cooperate. 

The evolutionary dynamics is defined as follows. The summation of the payoff over all 
the opponents defines the aggregated payoff to a player. The fitness, i.e., the propensity to 
reproduce, of a player is a linear function of the payoff. The proportionality constant controls 
the intensity of selection. At the end of each generation, a single player whose strategy (i.e., 
C or D) is replaced is selected with the equal probability from the population. If the selected 
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player is ordinary player, a parent, which is either ordinary or zealous player, is selected from 
the entire population with the probability proportional to the fitness. Then, the strategy of 
the updated player is replaced by that of the parent player. This update process is equivalent 
to the Moran process, a standard model of the birth-death process (e.g., [2]). If the updated 
player is zealous player, its strategy turns to C with probability p. With probability l—p, the 
updated player obeys the rule used by the ordinary player to adopt the strategy of a parent 
selected with the probability proportional to the fitness. 

I normalize the density of the ordinary players to unity and denote the added density of 
zealous players by y(> 0). The densities of cooperators among the ordinary and zealous players 
are denoted by x c (0 < x c < 1) and y c (0 < y c < y), respectively. The mean fitness, with the 
divisive factor l + y (i.e., the total population density) intentionally neglected, is defined by 

(tt) = (x a + Vc)^c + (1 - x c + y - y c )^o, (5) 

where 

%c = i _ w + w t( Xc + y^ R + (i-xc + y- y c )S] 

l + y 

and 

w [(x c + y c )T + (l-x c + y- y c )P] 

7T D = 1 - W + — (7) 

l + y 

are the fitness to a C and D player, respectively, and w (0 < w < 1) indicates the intensity 
of selection [271|28]. Equations ()6]) and ([7]) indicate that the payoff to a player per opponent 
is translated to the fitness with proportionality constant w. I assume (tt) > such that the 
selection of the parent player with the probability proportional to the fitness is well defined. 

In the reproduction phase, a cooperator and defector are selected as parent with probability 
(x c + y c )^cl (7r) an d (1 — x c + y — y c )T<~ D / (tt), respectively. Therefore, the dynamics of the 
fraction of cooperators among the ordinary players is given by 

[(^c + Z/cKc(l - x ) - (1 - x c + y - y c )7r D a: c ] . (8) 



dx c 1 



dt (it) 

10 



When y — 0, Eq. OH]) is equivalent to the meanfield equation of the Moran process. If the 
divisive factor (n), which just controls the time scale of the dynamics in this special case, is 
neglected, Eq. (|S]) is reduced to the usual replicator dynamics. 

The dynamics of the density of cooperators among the zealous players is given by 



dye 



p(y - Vc) + 



i-p 



[(x c + y c )^c(y -yo)-(i-x a + y- yc^yd 



(9) 



d* " w " w ' (tt) 

When p — 1, zealous players always cooperate (i.e., y c = y). In this case, Eq. (jHJ), with (tt) in 
the denominator neglected, is equivalent to Example 2 given in [39J. 

Pairwise comparison rule 

In the so-called pairwise comparison rule, the probability that the replacement occurs depends 
on the difference between the payoffs to two randomly selected players. At the end of each 
generation, I randomly select two players from the population without bias. If the two players 
are both cooperators or both defectors, nothing takes place. Otherwise, C replaces D with 



probability 1/ 



I _|_ g-/3(Tc-TD) 



\ _|_ g-^(Trc-TD) 



1/ 



1 + e 



-/9(Vd— to) 



and D replaces C with probability 1 — 1/ 
The intensity of selection is controlled by /3(> 0). 
The evolutionary dynamics for the infinite population under the pairwise comparison rule 
is represented by 



dxc 
dt 



1 + y 



(1 - x c )(x c + y c ) x c {\ - x c + y - y c ) 



1 -\- g-^C^C-TD) 



1 -)_ e -/3(7TD-Tc) 



and 



dy c 
dt 



p(y - yc) + 



2(1 -p) 



(y - yc){x c + y c ) _ i/c(i -x c + y-y c ) 

1 -)- e -P(nc-icx>) 1 4- e^^^n-Tc) 



(10) 



References 



[1] Nowak, M. A. Five rules for the evolution of cooperation. Science 314, 1560-1563 (2006). 



11 



[2] Nowak, M. A. Evolutionary Dynamics (The Belknap Press of Harvard University Press, 
MA, 2006). 

[3] Fowler, J. H. & Christakis, N. A. Cooperative behavior cascades in human social networks. 
Proc. Natl. Acad. Set. USA 107, 5334-5338 (2010). 

[4] Yamagishi, T. & Cook, K. S. Generalized exchange and social dilemmas. Social Psychology 
Quarterly 56, 235-248 (1993). 

[5] Molm, L. D., Collett, J. L. & Schaefer, D. R. Building solidarity through generalized 
exchange: a theory of reciprocity. Am. J. Sociol. 113, 205-242 (2007). 

[6] Dufwenberg, M., Gneezy, U., Girth, W. & van Damme, E. Direct vs indirect reciprocity: 
an experiment. Homo Oecono. 18, 19-30 (2001). 

[7] Greiner, B. & Levati, M. V. Indirect reciprocity in cyclical networks — an experimental 
study. J. Econ. Psych. 26, 711-731 (2005). 

[8] Stanca, L. Measuring indirect reciprocity: whose back do we scratch? J. Econ. Psych. 
30, 190-202 (2009). 

[9] Boyd, R. & Richerson, P. J. The evolution of indirect reciprocity. Soc. Netw. 11, 213-236 
(1989). 

[10] Masuda, N. Clustering in large networks does not promote upstream reciprocity. PLoS 
ONE 6, e25190 (2011). 

[11] Nowak, M. A. & Roch, S. Upstream reciprocity and the evolution of gratitude. Proc. R. 
Soc. Lond. B 274, 605-610 (2007). 



12 



[12] Hamilton, I. M. & Taborsky, M. Contingent movement and cooperation evolve under 
generalized reciprocity. Proc. R. Soc. B 272, 2259-2267 (2005). 

[13] Rankin, D. J. & Taborsky, M. Assortment and the evolution of generalized reciprocity. 
Evolution 63, 1913-1922 (2009). 

[14] Barta, Z., McNamara, J. M., Huszar, D. B. & Taborsky, M. Cooperation among non- 
relatives evolves by state-dependent generalized reciprocity. Proc. R. Soc. B 278, 843-848 
(2011). 

[15] Iwagami, A. & Masuda, N. Upstream reciprocity in heterogeneous networks. J. Theor. 
Biol. 265, 297-305 (2010). 

[16] van Doom, G. S. & Taborsky, M. The evolution of generalized reciprocity on social 
interaction networks. Evolution 66, 651-664 (2012). 

[17] Centola, D., Wilier, R. & Macy, M. The Emperor's Dilemma: A computational model of 
self-enforcing norms. Am. J. Sociol. 110, 1009-1040 (2005). 

[18] Wilier, R., Kuwabara, K. & Macy, M. W. The false enforcement of unpopular norms. Am. 
J. Sociol. 115, 451-490 (2009). 

[19] Traulsen, A., Semmann, D., Sommerfeld, R. D., Krambeck, H. J. & Milinski, M. Human 
strategy updating in evolutionary games. Proc. Natl. Acad. Sci. USA 107, 2962-2966 
(2010). 

[20] Kirchkamp, O. & Nagel, R. Naive learning and cooperation in network experiments. Games 
Econ. Behav. 58, 269-292 (2007). 

[21] Mobilia, M. Does a single zealot affect an infinite group of voters? Phys. Rev. Lett. 91, 
028701 (2003). 

13 



[22] Mobilia, M., Petersen, A. & Redner, S. On the role of zealotry in the voter model. J. Stat. 
Mech. P08029 (2007). 

[23] Galam, S. & Jacobs, F. The role of inflexible minorities in the breaking of democratic 
opinion dynamics. Physica A 381, 366-376 (2007). 

[24] Xie, J. et al. Social consensus through the influence of committed minorities. Phys. Rev. 
E 84, 011130 (2011). 

[25] Singh, P., Sreenivasan, S., Szymanski, B. K. & Korniss, G. Accelerating consensus on 
coevolving networks: The effect of committed individuals. Phys. Rev. E 85, 046104 (2012). 

[26] Coleman, J. S. Free riders and zealots: The role of social networks. Sociol. Th. 6, 52-57 
(1988). 

[27] Nowak, M. A., Sasaki, A., Taylor, C. & Fudenberg, D. Emergence of cooperation and 
evolutionary stability in finite populations. Nature 428, 646-650 (2004). 

[28] Ohtsuki, H., Hauert, C, Lieberman, E. & Nowak, M. A. A simple rule for the evolution 
of cooperation on graphs and social networks. Nature 441, 502-505 (2006). 

[29] Boyd, R. & Richerson, P. J. Culture and the Evolutionary Process (The University of 
Chicago Press, Chicago, 1985). 

[30] Szolnoki, A., Wang, Z. & Perc, M. Wisdom of groups promotes cooperation in evolutionary 
social dilemmas. Sci. Rep. 2, 576 (2012). 

[31] Mobilia, M. Stochastic dynamics of the prisoner's dilemma with cooperation facilitators. 
Phys. Rev. E 86, 011134 (2012). 



14 



[32] Blume, L. The statistical mechanics of strategic interaction. Games Econ. Behav. 5, 
387-424 (1993). 

[33] Szabo, G. & Toke, C. Evolutionary prisoner's dilemma game on a square lattice. Phys. 
Rev. E 58, 69-73 (1998). 

[34] Traulsen, A., Nowak, M. A. & Pacheco, J. M. Stochastic dynamics of invasion and fixation. 
Phys. Rev. E 74, 011909 (2006). 

[35] Maynard Smith, J. Evolution and the Theory of Games (Cambridge University Press, 
Cambridge, UK, 1982). 

[36] Sugden, R. The Economics of Rights, Co-operation and Welfare (Blackwell, New York, 
NY, 1986). 

[37] Hauert, C. & Doebeli, M. Spatial structure often inhibits the evolution of cooperation in 
the snowdrift game. Nature 428, 643-646 (2004). 

[38] Oliver, P. E. Formal models of collective action. Annu. Rev. Sociol. 19, 271-300 (1993). 

[39] Chatterjee, K., Zufferey, D. & Nowak, M. A. Evolutionary game dynamics in populations 
with different learners. J. Theor. Biol. 301, 161-173 (2012). 

Acknowledgments 

I thank Mitsuhiro Nakamura and Shoma Tanabe for careful reading of the manuscript and 
Masanori Takezawa for valuable discussions. This work is supported by Grants-in-Aid for 
Scientific Research (Nos. 20760258 and 23681033, and Innovative Areas "Systems Molecular 
Ethology" (No. 20115009)) from MEXT, Japan. 



15 



Additional information 

Competing financial interests: The author declares no competing financial interests. 



16 



(a) (b) 




Figure 1: Fraction of cooperators among the ordinary players in the presence of perfectly zealous 
cooperators (i.e., p = 1). The lines represent T = 1 + y/w. I used a typical payoff matrix of 
the prisoner's dilemma game given by R = 1, T > 1, and S = P = 0. I set (a) w = 0.1 and (b) 
w = 1. 
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Figure 2: Fraction of cooperators among the ordinary players as a function of the additional 
density of zealous players (i.e., y) and the probability of unconditional cooperation for zealous 
players (i.e., p). I set R = 1 and S = P = 0. (a) w = 0.1, T = 1.5. (b) w = 0.1, T = 2.5. (c) 
w = 1, T = 1.5. (d) w = 1, T = 2.5. 
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Figure 3: Fraction of cooperators among the ordinary players in the prisoner's dilemma game 
when the pairwise comparison rule is used for the updating. I set R = 1, S — P — 0, and 
/3 = 0.5. (a) T = 1.5. (b) T = 2.5. 
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Figure 4: Fraction of cooperators among the ordinary players as a function of the temptation 
payoff (i.e., T). I set R = 1 and S = P = 0. (a) w = 0.1. (b) w = 1. 
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Figure 5: Fraction of cooperators among the ordinary players as a function of y and p in 
snowdrift game. I set R = p - 0.5, S = p - 1, T = 0, P = 0, and /3 = 0.5. (a) w = 0.1. 
w = 1. 
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